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In this paper, we derive exponential bounds on probabilities of 
large deviations for "light tail" martingales taking values in finite- 
dimensional normed spaces. Our primary emphasis is on the case 
where the bounds are dimension-independent or nearly so. We demon- 
strate that this is the case when the norm on the space can be ap- 
proximated, within an absolute constant factor, by a norm which is 
differentiable on the unit sphere with a Lipschitz continuous gradi- 
ent. We also present various examples of spaces possessing the latter 
property. 



1. Introduction. It is well-known that for a sequence of independent 
zero mean random reals with light tail distributions (e.g., such that 

E |exp{|,^j|°crj~"}| < exp{l} for certain a G [1, 2] and deterministic at > 0), 
a "typical magnitude" of the sum St = J2i=i is "at most of order of 
S;^", meaning that 



Prob<^ > [1 + 7] 



<0(l)exp{-0(l)7'^} 



1=1 



for all 7 > 0; here in what follows, all 0(1) are positive absolute constants. 
The question we focus on in this paper is to which extent the above large 
deviation bound is preserved when passing from scalar random variables to 
independent zero mean random variables taking values in a normed space 
{E, II • II) of (possibly, large) dimension n < oo. Now our "light tail" condition 
reads 

(1) E{exp{||e.|r^r}}<exp{l} 
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for some a € [1,2], and what we want to get is a bound of the form 



V7>0:Prob<^ ll^^ill > [0 + 7] 



<0(l)exp{-0(l)7"} (*) 



i=l 



with a "moderate" value of the constant 6. It is immediately seen that our 
goal is not always attainable. For instance, let {E, \\ ■ ||) be ii (i.e., 
equipped with the norm ||a;||i = J27=i 1^*1)) ^^'^ take values itej with 

probability 1/2, 1 < i < n, where are the standard basic orths in R"'. 
Then ([1]) holds true with cij = 1, while H^fcHi = k whenever k < n. We 
see that in order for (*) to be true, 6 should be as large as 0{l)y/n. On 
the other hand, with 9 = 0{l)VdimE, {*) indeed is true independently of 
the norm || • || in question (see Example 13.11 in Section [3.ip . Our major goal 
in this paper is to show that a sufficient condition for (*) to be valid with 
certain 9 is 9^ -regularity of the space {E, \\ ■ ||). The latter means, essentially, 
that II • II can be approximated within an absolute constant factor by a norm 
p(-) which is continuously differentiable outside of the origin and possesses 
Lipschitz continuous, with the Lipschitz constant 9"^, derivative on its unit 
sphere: 



(2) 



p{x) = p{y) = 1 ^ P*{p'{x) - p'{y)) < 9^p{x - y) 



(here p.^ is the norm on the dual space E* , which is dual to p) . Examples of n- 
regular norms with "moderate" k include the spaces (R", || • ||p) {Lp on an n- 
point set with unit masses of points) and the spaces (R™^'", |-|p), 2 < p < 00, 
of m X n matrices with the Shatten norms \X\p = \\a{X)\\p, cr{X) being the 
vector of singular values of a matrix X; in both cases, p E [2, 00]. The 
spaces of the first series are K-regular with k = 0(1) min[p, ln(n + 1)], while 
the spaces of the second series are K-regular with k = 0(1) min[p, ln(m + 
l),ln(n+l)]. 

Norms p{-) satisfying ([2]) play important role in the theory of Banach 
spaces (where they are called norms with smoothness modulus of power 2). 
In particular, a number of results on the properties of martingales taking 
values in Banach spaces with smooth norms (see, e.g., Oil]) are available. 
However, we were unable to locate in the literature a result equivalent to 
Theorem 12.11 which establishes the validity of (somehow refined) bound (*) 
in the case of a 0^-regular space {E, \\ ■ ||). Thus, the main result of this paper, 
to the best of our (perhaps incomplete) knowledge, is new. The preliminary 
and slightly less accurate, version of Theorem 12.11 was announced in [10] and 
proved in the preprint [llj . 
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While the question we address seems to be important by its own right, our 
interest in it stems mainly from various applications of (somehow rudimen- 
tary) bounds of type (*) we have encountered over the years. These appli- 
cations include investigating performance of Euclidean and non-Euclidean 
stochastic approximation [Tj [5] , nonparametric statistics [SI [U [9] , optimiza- 
tion under uncertainty [TO] , investigating quality of semidefinite relaxations 
of some difficult combinatorial problems [12], etc. 

Our paper is organized as follows: the main result on large deviations 
(Theorem 12. ip is formulated in Section [2l Section 13.11 contains instructive 
examples and characterizations of K-regular spaces, along with a kind of 
"calculus" of these spaces. All proofs are placed in the appendix. 

In what follows, if not explicitly stated otherwise, we suppose all the 
relations between random variables to hold a.s.. 

2. Main result. 

2.1. Regular spaces. We start with the following 

Definition 2.1. Let (E, \\ ■ ||) be a finite- dimensional normed space and 
let K > 1 . 

(i) The function p{x) = \\x\\'^ called K-smooth if it is continuously differen- 
tiable and 

(3) Vx, y e E : p{x + y) < p{x) + Dp{x)[y] + Kp{y). 

(ii) Space {E, \\ ■ ||) (and the norm \\ ■ \\ on E) is called K-regular, if there 
exists G [1, k] and a norm || • ||+ on E such that {E, \\ ■ || + ) is n^-smooth 
and II • 11+ is k,/ k^- compatible with \\ ■ \\, that is, 

(4) yx e E -.Wxf <\\x\\l < —\\xf. 

(iii) The constant k{E, \\ ■ \\) of regularity of E, \\ ■ \\ is the infinum (clearly 
achievable) of those k, > 1 for which {E, \\ ■ \\) is K-regular. 

As an immediate example, an Euclidean space (R", || • II2) is 1-smooth 
and thus 1-regular. 

2.2. Main result. Assume that we are given 

• a finite-dimensional space {E, \\ ■ ||), 

• a Polish space with Borel probability measure fi, and 
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• a sequence !Fq = {0, il} C J^i C ^2 C ... of a-sub-algebras of the Borel 
cr-algebra of 0. 

We denote by Ej, i = 1,2, ... the conditional expectation w.r.t. J^i, and by 
E = Eq the expectation w.r.t. /i. 

We further assume that we are given an iiJ-valued martingale-difference 
sequence = {Ciji^i of Borel E'- valued functions on 17 such that is 
^j-measurable and 

E,_l{e^} = 0, i = l,2,... 

An immediate consequence of Definition 12.11 of the regular norm is as fol- 
lows: assume that an valued martingale-difference ^ = {Cilt^i is square- 
integrable: E < cr^ < 00. Then 

n 

E{||5Jp}<«^a?. 

i=l 

Indeed, || • ||+ is K+-smooth, we have 

p{St+i) < p{St) + Dp{St)[it+i\ + 

whence, taking expectations and making use of the fact that ^ is a martingale- 
difference, 

E{p(5t+i)} < EMSt)} + K+E{p(em)} < E{p(5t)} + kE{ 116+1 IP} 
by the right inequality of Then, by the left inequality of ([3]), 

n n 

E{||5„f} <e{||S„||^} <K^E{||eif} <K^a2. 

t=i t=i 

Our primary objective is to establish exponential bounds on the probabilities 
of large deviations for an valued martingale difference {6}. To this end, 
we impose on a "light tail" assumption as follows. Let a G [1,2] and 
a sequence cj°° = {cjj > 0}^^ of (deterministic) positive reals be given. We 
introduce the following condition on the sequence 

\/i>l: E,_i {exp{||6||"cJi~"}} < exp{l} almost surely (Ca[a°°]) 

Our main result is the large deviation bound for Sn = Y^f=i ii as follows: 

Theorem 2.1. Zei (i?, ||-||) hen-regular, let E -valued martingale- difference 
satisfy (Ca[o-~]), and let Sn = T.f=i ii, <^'^ = Wi; o"iv]- Then 
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(i) for 1 < a < 2, one has for all N > 1 and 7 > 0/ 
2eK + V27 



(5) 

Prob<' IIS^II > 



N 



Y.a'i) < 2exp<j -— mill 



1 



2. 2-an 

7 ;7* 7 



where 
(6) 



7* = 7*(,a,i/ 



8«t 


2-a 




> 16 




2"* 









32 

limQ,^i+o7*(a, '^^) = 
limQ,_2-o7*(a)i^^) = +00, 

(ii) When a = 2, the hound (jl2|) improves to 



> 16, 



1 < a < 2, 

a = 1, 
a = 2. 



(7) 



(ViV > 1,7 > 0) : Prob<^ H^^vll > \^ + V2- 



N 



^^^2^ <exp{-7V3}. 



(iii) When the condition Ej_i |exp{||^j|pcj. ^}| < exp{l} in (C2[o"°°]) is 
strengthened to < ai almost surely, i = 1,2, the bound ()12|) improves 
to 
(8) 



(ViV> 1,7 >0) :Probi \\Sn\\ > \V2^ + V2' 



7 



N 



< 



exp 



{-7^2} . 



3. Regular spaces. To make Theorem 12.11 meaningful, we need to 
point out a spectrum of interesting K-smooth/regular spaces, and this is 
the issue we consider in this Section. 



3.1. Basic examples. Let E be an n-dimensional hnear space, and let || • || 
be a norm on E. It is well known [2] that there exists an ellipsoid Q centered 
at the origin such that Q C {x G E : \\x\\ < 1} C ^JnQ, or, equivalently, 
there exists a Euclidean norm || • 11+ on E such that ||a;p < < n||x|p. 

Since the Euclidean space {E, || • ||+) is 1-smooth, we conclude that 

Example 3.1. . Every finite- dimensional normed space [E, ||-||) is (dimi?)- 
regular. 

We are about to present a number of less trivial examples, those where 
the regularity parameter k is dimension-independent (or nearly so). 
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Example 3.2. Let 2 < p < oo. The space (R", || • ||p) with n > 3 is 
Hp{n) -regular with 

2 2 

(9) f^pin) = mill (p — l)np p < min[p — l,21n(n)] 

2<p<p 
p<oo 

Example 3.3. Let 2 < p < oo. The norm \X\p = \\a{X)\\p on the space 
j^mxn of m X n real matrices, where a{X) is the vector of singular values 
of X , is Kp{m,n)-regular, with 



(10) ' ^^'^^ 



mill max[2, p — l](min(?7T,, n)) 



2 2 
P P 



< mill [max[2,p — 1], (2 ln(min[m, n] + 2) — 1) exp{l}] . 



The proof of the bound (|T0|) is based upon the fact which is important 
by its own right: 

Proposition 3.1. Let A be an open interval on the axis, and f he a CP' 
function on A such that for certain ^±,/i± G R one has 

Z< . A) : .^/>)±m+,_ < /'(")-/'(■■) < ;/"(^+,, 

Let, further, A'„(A) he the set of all n x n symmetric matrices with eigen- 
values belonging to A. Then A'„(A) is an open convex set in the space S" of 
n X n symmetric matrices, the function 

F{X) = Tr(/(X)) : A'„(A) ^ R 

is CP, and for every X G A'„(A) and every H £ one has 
(12) 

e^TT{Hf"{X)H)-^lj.^Tr{H^) < D^F{X)[H,H] < e+Tr{Hf"{X)H)+fi+TT{H'^). 

3.2. Dual characterization of smoothness and regularity. The fohowing 
well-known fact can be seen as dual characterization of Ac-smoothness: 



Proposition 3.2. Let {E, \\ ■ ||) be a finite- dimensional normed space, 
E* he the space dual to E, || • ||* he the norm on E* dual to \\ ■ ||; and let 

x) stand for the value of a linear form £ E* on a vector x £ E. Let 
also f{x) = l\\xf : E and /*(0 = ^||^||* : E* ^ K. The following 

properties are equivalent to each other: 

(i) {E, II • II) is K-smooth; 
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(ii) df{x) = {/'(a;)} is a singleton for every x, and 

(13) (/'(x) - f{y),x -y)< k\\x - yf Vx, y G E; 

(iii) / is continuously differentiable, and /'(•) is Lipschitz continuous with 
constant k: 

(14) <^||^_y|| yx,yeE; 

(iv) One has 

y{^,r,eE*, xedMO, yedMr,)) - rj,x - y) > k-^^ - rj\\l; 

(v) One has 

y{^,r,eE*, xedMO, y&dfM)- \\x - y\\> t^-^U- 

(vi) One has 

y{^,r,eE,, xedMO) : M^ + v)>MO + {v,x) + :^\\v\\l 

In 

Another characterization of regular spaces is as follows: 

Proposition 3.3. Let {E, \\ ■ ||) be a finite- dimensional normed space, 
E* be the space dual to E, || • ||* be the norm on E* dual to \\ • \\, and let 
stand for the value of a linear form ^ ^ E* on a vector x ^ E. Let 
also B^: be the unit \\ ■ ||*-6a// of E* . 

(i) If{E, II • II) is K-regular, then the exists a continuous function V : ^ 
R which is strongly convex, with coefficient 1 w.r.t. || • ||*, on B^,, that is, 
possesses the following equivalent to each other properties: 
(15) 

(a) 1] £ intB^, X £ dv{^),y £ dv{r])) : - v,^ - v) > U - vWl, 

2 



(b) y{^,ri: ^,^ + riemtB,, xedviO) : + i^) > v{0 + {v,x) + 



and, in addition, is such that 

(16) maxf — minv < — 

(ii) Assume that the unit ball B^ of (E* , \\ ■ ||*) admits a function v satisfy- 
ing (jlSp . (I16p . Then (E, \\ ■ \\) is 0[\)K-regular with an appropriately chosen 
absolute constant 0(1). 
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3.3. "Calculus" of smooth and regular spaces. 

Proposition 3.4. Let (E, \\ ■ \\e) be a finite- dimensional normed space, 
L be a linear subspace of E, and F = E/L be the factor-space of E equipped 
with the factor-norm = ™iii/g/ ll/ll-B- If {EA\ ' We) is K-smooth (k- 

regular), then (L, \\ ■ \\e) and (F, \\ ■ \\f) also are n-smooth, respectively, k- 
regular. 

Proposition 3.5. (i) Letp G [2,oo], and let {Ei, \\-\\i) be finite- dimensional 
K-smooth spaces, i = l,...,m > 2. The space E = Ei x ... x E^ equipped 
with the norm 

(the right hand side is max when p = oo) is k'^ -regular with 

i 

1 2 _ 2 

(17) K = min[/« + p— l]mf p < min[K +p — 1, [k + 2 ln(m) — 1] exp{l}]. 

2<p<p 

(ii) Let II • ||j be n-smooth norms on E. Then the norm 



i=l 



is mK-regular on E. 



Proposition 3.6. (i) Letp G [2,oo], and let [Ei, be finite- dimensional 
K-regular spaces, i = l,...,m > 2. The space E — E\ X ... X Em. equipped 
with the norm 

/ m \ i/p 

„1 ^mMI _ ( l|x*||^ I 



ix\...,xn\\ = {Y: 



\i=l / 

(the right hand side is max ||x*||j when p = oo) is k'^^ -regular with 
(18) 

II 1-1 

= 2 mill [k + yO — l]mp p < 2min[K -\- p — 1, [k + 21n(?n) — 1] exp{l}]. 

2<p<p 

(ii) Let II • ||i be K-regular norms on a finite- dimensional space E. Then 
the norm 

m 

ll-^ll — ^ ^ 
i=l 

is 2mK-regular on E. 
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4. Appendix: Proofs. 

4.1. Proofs for Section \3.1 



4.1.1. Justifying the Examples. 

Example \3.S^ . Let 2 < p < co. We claim that in this case the space (-R"", || • 
lip) is l)-smooth. Indeed, the function = ||-||p is convex, continuously 
difFerentiable everywhere and twice continuously difFerentiable outside of the 
origin; for such a function, ^ holds true if and only if 

(19) D'^p{x)[h, h] < 2K+p{h) y{x, heE,x^O); 

since p{-) is homogeneous of degree 2, the validity of (jl9p for all x,h is 
equivalent to the validity of the relation for all h and all x normalized by 
the requirement p{x) = 1. Given such an x and h and assuming p > 2, we 
have 

Dp{x)[h] = 2 {j:\xi\p)'' j:\xi\P-hignix^)h 

——2 / \ 2 

DM^)[h,h] = 2(--l) (E\xiK'sign{x,)hi 




P^-P-^ Y.{p - l)\x^\P-''h1 < 2{p - 1) E \xi\P-''h'i 



1 

p-2 2 



< 2{p-i)(Ei\x.r')~ {n\h. 



= 2{p - i)\\hr^ = 2{p - i)p{h) 

as required in (jl9p when k+ = p — 1 . In the case of p = 2 relation (|19p with 
= p — 1 = 1 is evident. 

2 2 

Now, when p G [2,p] and x G R", one has ||2;||p/||x||p € [l,n'p p], so that 

2 _ 2 

(R", II • lip) is K-regular with k = (p — l)np f, and ([9]) follows. ■ 
Example \ 3. 31 . 1^. We start with the following 



Lemma 1. Let p > 2. Then the space S" of symmetric n x n matrices 
with the norm \X\p is n-smooth with 

(20) K = max[2,p- 1]. 



imsart-aap ver. 2007/12/10 file: LD-5-5-2008.tex date: September 4, 2008 



10 



A. JUDITSKY AND A. NEMIROVSKI 



Proof. The statement is evident when p = 2; thus, from now on we 
assume that p > 2. Let us apply Proposition 13.11 to A = R, f{t) = \t\P 



with 0^ 



0, /i+ = and 



max 



1 



(this choice, as it is 



immediately seen, satisfies (jlip ). By Proposition, the function F{X) = \X\p 

on S" is twice continuously differentiable, and 

(21) 

yX,H ■.0< D^F{X)[H,H] < 9+Tr{f"{x)H^), 9+ = max 



p-l 



It follows that the function p{X) = |X|p = {F{X))p is continuously dif- 
ferentiable everywhere and twice continuously differentiable outside of the 
origin. For X / we have Dp{X)[H] = '^{F{X))~p~^ DF{X)[H], whence 
(22) 

X^Q^D^p{X)[H,H] = l 



{F{X))-\DF{X)[H]Y + l{F{X))-'D^F{X)[H,H] 



<o 



<l{F{X))-p^^e+T,{f"{x)H^). 

1 -—1 

Setting Z = -^j^j^^{F{X)) p f"{X), p = -j^, it is immediately seen that 



\Z\p = 1. From (j22p we have 
(23) 

D^p{X)[H,H] < 2G+(p- l)Tr(Zii'2) < 2e+{p - l)\Z\p\H^\^ = 2e+{p - 1)\H^\e 
= 29+{p-l)\H\l. 

Now, if X, y G S" are such that the segment [X; X + Y] does not contain 
the origin, then 

37 G (0, 1) : p(X + y ) < p{X) + Dp{X) [Y] + \d''p{X + 7^) [Y, Y] , 



and (123p implies that for the outlined X, Y one has 

p{X + Y)< p{X) + Dp{X)[Y] + e+{p - l)p{Y). 

Since p is C^, the resulting inequality, by continuity, is valid for all X,Y .» 
2". Now we can complete the justification of Example l3.31 W.l.o.g. we may 

assume that m < n. Given an m x n matrix X, let S{X) 

gm+n_ Qj^g clearly has 

\\<y{X)\\, = \X\, = 2-'/P\S{X%, 





X ' 
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whence, by Lemma [T] and due to the fact that the mapping X S{X) : 
Y^rnxn _^ gm+n ^i^ear, the norm | • |p, treated as a norm on R*"^", is 
max[2,p— l]-smooth whenever p>2. Since a{X) e for X G R"^^", for 
every p G [2, oo) such that p < p one has 

\X\l<\X\l<m7-l\X\l. 

2 _ 2 

Thus, the space (R™^", | • L) is K-regular with k = min max[2, p— l]mp p , 

2<p<oo 

P<P 

and we arrive at (IIOD. ■ 



4.1.2. Proof of Proposition [3!7l Let {/fc(t)} be a sequence of polynomials 
converging to /, along with the first and the second derivatives, uniformly 



on every compact subset of A. For a polynomial p{t) = J2jLoPj'l^'^ the func- 
tion P{X) = Tv{J2jPjX^) is a polynomial on S". Let now X,H e S", let 
As = \s{X) be the eigenvalues of X, X = U Diag{X}U^ be the eigenvalue 
decomposition of X, and let H be such that H = UHU^ . We have 
(24) 

P{X) = Es=iP(A.(X)) (a) 
DP{X)[H] = T\^(Ef=iEf=o'^'^^^-'-'=Tr(p'(X)i/) = ELiP'(As(^))^.. {h) 

Further, let 7 be a closed contour in the complex plane encircling all the 
eigenvalues of X. Then 



DP{X)[H] = Ti{p'{X)H) = fp'iz) Tt{{zl - X)-^H)dz 



1 



D^P{X)[H, H] = ^^ §p\z) Tr((z/ - Xr^H{zI - Xr^H)dz = ^ § Elt=i Jl^f^dz. 
Computing the residuals, we get 

f p'{\s)-p'{\t) ^ _L X 

(25) z)2p(x)[i7,i7]=^r,,,[p]F2„ r,,,[p]= A A. ' 



s± 



p"iXs), Xs = X 



Substituting p = fk into (p^ a. b) and ([25]) . we see that the sequence of 
polynomials Fk{X) = Tr(/fc(X)) converges, along with the first and the 
second order derivatives, uniformly on compact subsets of ^^^(A); by (j24[ a). 
the limiting function is exactly F{X). We conclude that F{X) is on 
A'„(A) and that the first and the second derivatives of this function are 
limits, as A; — > 00, of the corresponding derivatives of Ffc(X), so that for X = 
UBiag{X}U^ £ ^n(A) (where U is orthogonal) and every H = UHU'^ G S" 
we have 



DF{X)[H] = Y.snK)Hss = TT{nX)H) 
^ > D^F{X)[H,H] = Es^t^sAml 
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So far, we did not use (jlip . Invoking the right inequahty in (llip . we get 

D^F{x) [H, H] < Es,t [e+ ^"^''V'^''^ + ^'+] Hi = 0+ Es /"(A.) Et Hi + ^l+ Es,t Hi 

= 6+ TV(Diag{r(Ai), /"(A„)}^2) + ^(jj2) ^ Tvif"{X)H^) + Tr(//2), 



which is the right inequaUty in (jl2p . The derivation of the left inequahty in 
(1121) is similar. ■ 



4.1.3. Proof of Proposition \3.B . 

(i)=^(iii). : We are in the situation when / is continuously differentiable. 
Convolving /(•) with smooth nonnegative kernels 6k{-) with unit integral and 
support shrinking to origin as k ^ oo, we get a sequence /fc(-) of smooth 
functions converging to /(•), along with first order derivatives, uniformly on 
compact sets. We have 

fk{x + y) = Jf{x-z + y)6{z)dz<J[fix-z) + {f'ix-z),y) + Kf{y)]6iz)dz 
= fk{x) + {f'k{x),y) + Kf{y) 

From the resulting inequality combined with smoothness and convexity of 
fk it follows that 

< D^fk{x)[h,h] < K\\hf Vx,/i G E. 
Thus, if \\h\\ = \\d\\ = 1, then 

4:D'^fk{x)[h,d] = D'^fk{x)[h+d,h+d]-D^fk{x)[h-d,h-d] < K\\h+df < 4k 

. Whence D'^ fi^{x)[h, d] < k whenever \\h\\ = \\d\\ = 1, or, which is the same 
by homogeneity, 

\D^fk{x)[h,d]\<K\\h\\\\d\\ yx,h,d. 

Consequently, 

1 1 
\{f'kiy)-f'kix),h)\ = I J D^fk{x+t{y-x))[y-x,h]dt\ < J K\\y-x\\\\h\\dt < K\\y- 



whence, taking maximum over h with \\h\\ = 1, 

Wfkiy) - fkix)\\* < 

. As /e — > oo, /^(x) converge to /'(x), and we conclude that /'(•) possesses 
the required Lipschitz continuity. ■ 
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(iii)^(ii):. evident 

(ii) ^(i):. A convex function on with a singleton differential at every 
point clearly is continuously differentiable, so that in the case of (ii) / is 
continuously differentiable. Besides this, in the case of (ii) we have 

f{x + y) = f{x) + {f{x),y)+]\nx + ty)- f'{x),y)dt 



< f{x) + {nx),y) + / KtM^dt = f{x) + (/'(x), y) + Kf{y), 



which immediately implies ([3]) (recall that || • p = 2/(-)). ■ 

(iii) <^(v):. The functions /(•), /*(•) are the Legendre transforms of each 
other, so that x £ 5/*(^) if and only if G df{x). Now let (iii) be the case, 
and let £,,r] e E* and x G (9/*(0) U £ <9/*(??)- Then ^ = /'(x), rj = f'{y) and 
therefore, due to (iii), 

\\i - r/IU < k||x - 

so that (v) takes place. Vice versa, let (v) take place, and let x, y S 
^ G df{x), 7] G df{y). Then x G (?/*(^), y G df^,{y), and therefore (v) says 
that 

11^ - v\\* < - y\\- 

We conclude that if x = y, then ^ = ??, that is, df{x) always is a singleton, 
meaning that / is continuously differentiable, and that the inequality in (iii) 
takes place, that is, (iii) holds true. ■ 

(iv) ^(iu):. Let (iv) take place. If there exists x € E such that df{x) is 
not a singleton, then, choosing C^rj £ df{x) with ^ / ry, we would have 
X G 9/*(^), X G df^:{r]), whence by (iv) we should have 

- ?7,x - x) > - r]\\l, 

which is impossible. Thus, df{x) is a singleton for every x, so that / is 
continuously differentiable. Besides this, with x,y € E and ^ = /'(x), i] = 
f'{y) we have x G 5/*(0, y G df^{rj), whence, by (iv), 

- ri,x - y) > - r]\\l. 

Since 

{i-r],x -y) < lie - r/||*||x - y||, 
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we get 

U-vUx-y\\>^^'U-ri\\l 

whence 

ll^-^ll* = ll/'(^)-/'(y)ll*<^lk-?/ll, 

and thus (iii) takes place. 

Now let (iii) take place, and let us prove that (iv) takes place as well, or, 
which is the same in the case of (iii), that {f'{x) — f'{y),x — y) > K~^\\f'{x) — 
f{y)f. Setting 

g{u)=f{u)-{f'iy),u-y), 
we get a continuously differentiable convex function on E such that 

\\g'{x)-g'{y)\U<K\\x-y\\ 
and g'{y) = 0. Due to these relations, 

g{y + h) < g{y) + ^\\hf 

for all h. Now let e G £^ be such that {g'{x),e) = \\g'{x)\\^ and ||e|| = 1. Due 
to 

Wa'iu) - g'{v)\\^ < k\\u-v\\, 

we have 



g{x e) < g{x)-{g{x), e) + -|| e 



|2 



= ff(x)-M^M+''M^Ml5j_M^. 

On the other hand, g attains its global minimum at y, so that 
9{x) ^ > g{x e) > g[y). 

We now have 

9iy) + '^\\hf > g{y + h)>g{x) + {g'{x),y + h-x) 
> giy)+^^^^^ + {g'ix),y + h-x), 

{g'ix),x- y) > + {g'ix),h) - 



whence 

(n'(x).x-v] > 

This inequality is valid for all h; setting h = MS£1]1±q^ the right hand side 
becomes MJ£B±^ Thus, 

ii5'(^)ii^ \\nx)-ny)\\i 



{f'{x) - f'{y),x -y) = {g'{x), x - y) > 
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(iv) =^(vi):. Let (iv) take place, let ^,r/ G E* and x G Setting 
it = i + 'Pit) = f*{Ct), < t < 1, we get an absolutely continuous 
function on [0, 1] with the derivative which is almost everywhere given by 
(/)'{t) = {r],xt), with xt G <9/*(6)- We have 

Jo 

= cj){0) + / {t], xt)dt = ,^(0) + / [(??, x) + {rj, Xt - x)]dt 
Jo Jo 

= 0(O) + (??,x)+ / t~\{i + tr])-i,xt-x)dt 
Jo 



= (t){0) + {ri,x) + —\\ri\\l = f40 + iv,^) + ttII^II* 

where the inequality is given by (iv) . We end up with the inequality required 
in (vi). ■ 



(vi)^(i): 
We have 



Let (vi) be the case, let x G and ^ G df{x), so that x G 9/*(0- 



max [(^ + r],x + y) - /^(^ + ??)] 

ri£E* 



< max 

■q&E* 



max 



+ + y) - /*(0 - {r},x) - —WiiWl 



(C,a; + y) + (r/,y)-/*(0-;rll'?ll* 



{i,x) - f*{0+{C,y) +max 

V ' ^ 



{v,y) - ir^lvWl 



f{x) + {C,y) + 



This relation along with the relation f{x + y) > f{x) + {C,y) implies that ^ is 
the Prechet derivative of / at x, whence / is convex and differentiable, and 
thus - continuously differentiable function on E which satisfies the inequality 

f{x + y)<f{x) + {f'{x),y) + ^\\yf. , 

We have proved that (i)<^(ii)44>(iii)44'(iv)<^(v) and (iv)=^(vi)=^(i), mean- 
ing that all 6 properties in question are equivalent to each other. ■ 



4.1.4. Proof of Proposition 
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(i) :. Let (E, \\ ■ ||) be K-regular, and let k+ G [1, k] and || • ||+ be such that 
{E, II • ||-|_) is K-smooth and (jH) holds true, and let || • ||+^,, be the norm on 
E* dual to II • II + ; note that 

(27) ^11 • 11^ < II • <\\-\\l 

due to (g]). Invoking Proposition [321 the function v{(,) = f ||?||^,+ : -B* ^ R 
satisfies 

V(^,7? G mtB^, X e dv{^), y G dv{r])) : - ri,x - y) > — 1|^ - 7?|| + 

and thus satisfies ()15[ a) due to ()27p . At the same time, 

max u — min u = — max ||£|R^<— , 
B, 2 U\\*<i ^' ~ 2' 

where the concluding inequality is due to (f27|) . (i) is proved. 

(ii) :. Let v{-) satisfy (fT5]) and (fT6]) : clearly, the function ^[u(^) ^)] — 
v{0) also satisfy these relations; thus, we can assume w.l.o.g. that v{S,) = 



v{—S,) and v{0) = 0. Let V be the Legendre transform of v{-] 



V(x) = max \(£,x) — v(x)] . 
||g||.<i 



, that is. 



By the standard properties of the Legendre transform, (jl5|) implies that V 
is a continuously differentiable convex function on E such that 

V'{x) = argmin x) — G -B* and ||V^'(x) — T^'(?/)||* < ||a; — y|| Vx, y. 

In addition, we clearly have V{x) = V{—x) and ||rE|| — | < V{x) < \\x\\ 
for all X by (jl6p . Convolving V with a smooth symmetric w.r.t. the origin 
nonnegative kernel with unit integral and small support and subtracting a 
constant to make function vanish at the origin, we see that for every e > 
there exists a C°° convex function W = on E such that for all x G £' 
one has 

(a) W,{x) = W,{-x), WM = 0; 
..o^ (b) ||x||-f-e<W^,(x)<||x||+e 

^ ^ (c) \\W'ix)\U < 1 

(d) < {W"{x)dx, dx) < \\dxf ydx G E. 
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Assuming e < k/10, let us set B = {x : W{x) < k}. Then i? is a closed 
convex set symmetric w.r.t. the origin and such that 



(29) 



9 5 
{x : \\x\\ < Yq^} C B C {x : \\x\\ < —k} 



due to p8l 6). B is the unit ball of certain norm r(x) on E; by ()29p we have 
(30) 

Setting L{x) = p^(x), observe that the function L is given by the equation 



-||x|| < Kr[x) < —\\x\ 
5 9 



V{x/^L{x)) = K. 



It follows immediately from the Implicit Function Theorem that L is C°° 
outside of the origin, and since this function is the square of a norm, it is 
therefore on the entire space. Let us compute the second order differential 
of L at a point x 7^ 0. Differentiating twice the equation specifying L, we 
get 



DL{x){dx\ 



2L 



D^L{x)[dx,dx] = 2L 



{W',dx) 
{W',x) ' 
{W',dx) 



{W',x) 



+ 



2LV2 
{W',x) 



{W" 



{W',dx) ' 
dx ——. — ^x 



{W',x) 



{W',dx) ' 
~ {W',x) 



where L = L(x), W' = W' {L~^''^x),W" = W"{L^^ 



/2. 



We claim that 

(31) X / ^ < D'^L{x)[dx,dx] < —\\dxf. 

Indeed, D^L(x)[dx, dx] is homogeneous of degree in x, so that it suffices 
to verify the required relation when L(x) = 1, i.e., when W{x) = k. In this 
case, the required bound is readily given by the expression for D^L combined 
with (j28[ c. d) and the following observations: (1) for x in question, we have 
{W',x) > W{x) - W{0) = K, and (2) ||x|| < by 
Setting ||x||+ = |Kr(x) and invoking (129p . we have 

(32) ll-f <||-|a<0(l)||-f, 

while from (I3ip it follows that the function /(x) = ||x||^ satisfies 
\\f'{x)-f\y)\U<Oil)K\\x-y\\, 
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which combines with (I32p to imply that 

\\f{x)-fiy)U^,<0{l)K\\x-y\U. 

Thus, {E, II • II) is 0(l)K-smooth, whence, by dMl), (E, \\ ■ \\) is 0(l)K-regular. 
■ 

4.1.5. Proof of Proposition \3m The fact that a subspace of a K-smooth/regular 
space equipped with the induced norm is K-smooth/regular is evident. As 
about the factor-space F = E/L, note that the space dual to (F, || • ||i?) is 
nothing but the subspace = ■ = OVx G L} in E* equipped 

by the norm induced by || • ||*. Now assume that (E, \\ ■ \\e) is K-smooth. 
By Proposition 13.21 it follows that || ■ ||* possesses property (iv) and there- 
fore its restriction on possesses the same property. Applying Proposition 
13.21 again, we conclude that {F, \\ ■ \\f) is K-smooth. We see that passing to a 
factor-space preserves K-smoothness, and since this transformation preserves 
also relations like (jl]), it preserves K-regularity as well. ■ 

4.1.6. Proof of Proposition \3.5[ 
(i):. To prove (i), let Pi{x^) = \\x'^\\f. 

A.. Let p £ [2, cxd) be such that p < p, and let r = p/2. Our local goal is 
to prove 

Lemma 2. The norm \\ ■ \\ on E = Ei x ... x Em defined as 

\\{x\...,xn\\ = mx%,...,\\x"'\\m)t 

is -smooth, with 

(33) = K + p — 2 

Proof. We have 

p(x\...,X-) = ||(||xii,...,||x-|U)||2 = ||(pi(xl),...,p„,(x-))||,. 

From this observation it immediately follows that p{-) is continuously differ- 
entiable. Indeed, p > 2, whence r > 1, so that the function ||y||,. is contin- 
uously differ entiable everywhere on TV^ except for the origin; the functions 
Pi{x^) are continuously differentiable by assumption. Consequently, p{x) is 
continuously differentiable everywhere on = Eix ...x Em, except, perhaps, 
the origin; the fact that p' is continuous at the origin is evident. 
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Invoking Proposition 13.2^ in order to prove Lemma [2] it suffices to verify 
that 

(34) \\p'{x)-p'{y)\l<2K+\\x-y\\ 

for all X, y. Since p' is continuous, it suffices to prove this relation for a dense 
m. E X E set of pairs x, y, for example, those for which all blocks E Ei in 
X are nonzero. With such x, the segment [x, y] contains finitely many points 
u such that at least one of the blocks is zero; these points split [x, y] into 
finitely many consecutive segments, and it suffices to prove that 

\\p'{x')-p'{y')\U<2K+\\x' -y'W 

when x', y' are endpoints of such a segment. Since p' is continuous, to prove 
the latter statement is the same as to prove similar statement for the case 
when x', y' are interior points of the segment. The bottom line is as follows: 
in order to prove (I34p for all pairs x, y, it suffices to prove the same statement 
for those pairs x,y for which every segment [x*,?/*] does not pass through 
the origin of the corresponding Ei . 

Let x,y be such that [x*,y*] does not pass through the origin of Ei, i = 
1, ...,m. Same as in the item "(i)^(iii)" of the proof of Proposition 13.21 for 
every i there exists a sequence of C°° convex functions {p*(-) > 0}^^ on Ei 
converging to Pi{-) along with first order derivatives uniformly on compact 
sets and such that 

(35) \Dyi{u')[h\ h']\ < 2K\\h'\\j y{u\ W G E,). 

Functions p*(n) = \\{p\{v}), ...,p*^{u^))\\r clearly are convex, C°° (recall 
that p\{-) > 0) and converge to p{-), along with their ffi'st order derivatives, 
uniformly on compact sets. It follows that 

1 

(36) {p'{y) -p'{x),h) = lim [ D'^p\x + t{y - x))[y - x,h]dt. 



Setting F{y I,..., ym) = y\+...+y'^,y > 0, we have p*(n) = F^p\{u^), ...,p\^{vJ^)). 
Now let u G [x, ?/], and let v £ E. We have 

Dp^{u)[v] = r^^F-r~\p{{u^), ..^pi.iu'^)) (j2r{pi{u')Y-^Dp\{u')[v']j 
D'p\u)[v,v] = ^ - l) FT-\p\{u'),...,pl{nn) (Er{pl{u^)Y-^Dpl{u^)[v']y 

<o 

+F7-i(p* (ui), ...,pU^n) E [{r - l){pt{u')Y-HDpi{u')[vi]y + {piin^^y-^D^pliu'Yv^v^ 

i 

< F-r-\p\{u^),...,pl{u^))j: [{r - l)(p*(n^))--2(^p^(^^)[^.])2 +2K(p*(nO)^-V^(^0] 

i 
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whence 
(37) 

< D^p\u)[v,v] 

i 

Taking into account that Pi{-) are bounded away from zero on [x,y] and 
that converge, along with first order derivatives, to pi{-) uniformly on 
compact sets as t ^ oo, the right hand side in bound ([37]) converges, as 
t — > oo, uniformly in n G [x,y] and v, \\v\\ < 1, to 



u 



t\\p 



r-l)\\u 



j||p-4 



iDp,iu')[v^])^ + 2k\\u 



MlP-2||^.||2 



By evident reasons, < 2||n*|| whence 

(38) 



,i\\P 



E 4(r - l)||n^||r^||^;,||2 + 2k\\u'\\^-^\\v'\\^^ 



[2p + 2k - 4] E \\u 



,i\\P 



2k+ 

When p > 2, we have 



p-2 



2 

P \ P 



Y.\\u 



i\\P 



p-2 



i\\P 



and (|38|) implies that ^{u,v) < 2K_|_||t;|p. This inequality clearly is valid for 
/9 = 2 as well. Recalling the origin of ^'(•, •), we conclude that for every e > 
there exists such that 

t>te,ue [x,y], \\v\\ < 1 ^ < D'^p\u)[v,v] < 2k+\\v\\'^ + e. 

The resulting inequality via the same reasoning as in the proof of item 
"(i)=^(iii)" of Proposition [312] implies that 

t>t^,ue [x,y] =^ \D'^p\u)[v,w]\ < (2k+ + e)||t;||||u;|| yv,w. 

In view of this bound and (j36p . we conclude that 

{p'{y)-p'{x),h) <{2K+ + e)\\y-x\\\\h\\ 

for all h, whence ||p'(y) — p'(a:)||* < (2K_|_ + e)||y — Since e > is arbitrary, 
we arrive at (IHl 
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B.. When p < p, we have 

\\{\\x%, ||x-|U,)||2 < \\i\\x%, < mM||(||xii, |k™|U)||2, 
which combines with Lemma [2] to imply that the norm in (i) is K-regular 

2 _ 2 

with K = [p + K — 2]mp p , for every p G [2,p], and (i) follows. 

(ii):. To prove (ii), consider the norm I (x"^, x™) I = + ... + ||x™ 

on X X ... X i?. As it is immediately seen, this norm is K-smooth. If, 
further, =X]||^*l|j5 then 

i 

||x||| < |xp < m||2;||| \/x e E X ... X E, 

whence || • ||-|- is mK- regular. The norm in (ii) is nothing but the restriction 
of II • III on the image of E under the embedding x i— > (x, of E into 

E X ... X E, and it remains to use Proposition 13.41 ■ 

4.1.7. Proof of Proposition \3.6[ 
A useful lemma.. We start with the following fact: 

Lemma 3. Let (E, \\ ■ \\) be a finite- dimensional n-regular space. Then 
there exists k- smooth norm || • ||+ on ii^ such that 

(39) \l{x £ E) : \\xf < \\x\\l < 2\\xf. 

Proof. By definition, there exists k+ G [1, k] and a norm 7r(-) on E which 
is K^^-smooth and such that 

V(x G E) : ||x|p < 7r^(x) < /i||x|p, p. = 

or, which is the same, 

(40) yCeE*:7Tl{()>ml>-<{0, 

where E* is the space dual to E and vr*, || • ||* are the norms on E* conjugate 
to TT, II • II, respectively. 

In the case of p < 2, let us take || • ||+ = vr(-), thus getting a K_|.-smooth 
(and thus - K-smooth as well) norm on E satisfying (139^ . Now let p > 2, so 
that 7 = l/{p - 1) G (0,1). Let us set q,{0 = VtWHW^^WW*, so 
that g*(-) is a norm on E*. We have 

(41) y^eE*: qliO > UWl > = IqUC). 
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Further, by Proposition 13.21 we have 

V(C,r? G E*, X G dTrliO) : 7rl{^ + ??) > 7rl{0 + {v,x) + —Tr^iv), 

whence, due to ||^ + r]\\1 > + y) for ah ^,77 and every y from the 
subdifferential D{^) of || • ||^ at the point ^, 

qUC + r])> qliO + {v,x + y) + —ttUt]) > + {r],x + y) + ^qUv) 

(note that 7r,(-) > q,{-) by (00])). Since di^H^ + D{i) = dqt{0 and ^ = 
{t,-\)K+ ^ h we get 

V(e,r7 G E\ze dqliO) ■ + v) > qliO + {v,z) + -qHv). 

By the same Proposition 13.21 it follows that the norm || • ||_|_ = q{-) on E 
such that q^{-) is the conjugate of q{-) is K-smooth. At the same time, (|4ip 
implies (139]) . ■ 

Proof of Proposition \3.(A is readily given by Lemma [3] combined with 
the corresponding items of Proposition 13.51 E.g., to prove (i), note that 
by Lemma [3] we can find K-smooth norms qi{-) on Ei such that qf{x'^) < 
\\x III < 2qf{x') for every i and all x* £ E^. Applying Proposition 13.51 (i) to 

the spaces {Ei,qi{-)), we get that the norm q{x^, ...,x"^) = yJ2 Qi{x^)j 

on El X ... X Em is K^-regular with k.^ given by (|17p . Taking into account 
the evident relation 

q\x\...,xn<\\{x\...,xnf<2q\x\...,xn 

and recalling the definition of regularity, we conclude that || • || is K"'~"*"-regular, 
as required. ■ 



4.2. Proof of Theorem \2.1[ 

4.2.1. Reduction to the case of a smooth norm. We intend to reduce 
the situation to the one where {E, \\ ■ ||) is K-smooth rather than K-regular. 
Specifically, we are about to prove the following fact: 

Theorem 4.1. Lei (E', ||-||) ben-smooth, let E -valued martingale-difference 
satisfy (Ca[cj°°]), and let Sn = J2iLi^i> = [fi; cat] • Then 
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(i) When 1 < a < 2, one has for all N > 1 and 7 > 0; 
(42) 



Prob|||5A.|| > 

where 
(43) 

7* = 7^=(a,cr^) 



exp{l}K + 7 



N 



^ < 2exp<j -— mill 



2, 2-an 

7 ;7* 7 



8a* 


2-a 


\ lk^l|2 1 


> 16 


r iWh 1 


2-a 


2"* 











32 

a* 

linw+0 7*(«,^^) = 16S^ 

II II 00 



> 16, 



limQ,_^2-0 7*(">Cr ) = +00 ; 

(ii) When a = 2, the bound (j42p improves to 



1 < a <2, 

a = 1, 
a = 2. 



(44) (ViV> 1,7 > 0) : Prob<^ ||Sjv|| > [V^ + 7] 



TV 



^5:^2^ <exp{-7V3}. 



(iii) When the condition Ej_i |exp{||^j|p(T,- ^jj < exp{l} in (C2[o"°°]) is 
strengthened to \\^i\\ < fjj almost surely, i = 1,2, the bound (|42p improves 
to 



(45) (V7V> 1,7 >0) :Prob<^ ||5jv|| > [^/^ + 7] 



^5:^2^ <exp{-7V2}. 



It is immediately seen that Theorem 14.11 implies Theorem 12.11 Indeed, if 
{E, II • II) is K-regular, by Lemma [3] there exists a norm || • ||+ on E such 
that {E, II • 11+) is K-smooth and ([39]) holds true. Setting ai = \f2ai, observe 
that (IMl) combines with (Cq[o-°°]) to imply that Ei„i {exp{||^i||^CT^"2}| < 
exp{l}. Applying Theorem I4.1i (i) to the K-smooth space (E, || • ||-|_) and Oi 
in the role of Oi and taking into account that ||5Ar|| < ||S'Ar|| + , we see that 
Theorem 12.11 (i) is an immediate corollary of Theorem 14.11 (i) , and similarly 
for Theorem 12.11 (ii-iii) . 



4.2.2. Proof of Theorem preliminaries. In the sequel, we need the 
following (essentially, well-known) fact. 

Proposition 4.1. Let ipi, i = 1,...,N, be Borel functions on ^ such 
that ipi is J^i-measurable, let a G [1,2], and let m, Ui > be deterministic 
reals. Assume that almost surely one has 



(46) E,_i{^,} < M,,Ei_i {expllV'.r/z.f }} < exp{l}, 1 < i < iV. 
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Then for every 7 > one has 

N N N 



(47) Prob I E V'i > E + ^\ E ^ ^ 2 exp{-^ min [7', 7*-"7°J } 

\^i=l 2=1 \ i=l j 



where 
(48) 



/ N\ 



32 



8a« 
2"* 



> 16 



> 16, 



limQ,_,i+o7*(a, i^^) = 16|j|^ 
\min^2-o^*{a,v^) = +00, 



1 < a < 2, 



To make the text self-contained, here is the proof. 

(P .. Till item 4*^ of the proof, we restrict ourselves with the case when 
1 < a < 2. Besides this, by evident homogeneity reasons we may assume 
w.l.o.g. that V = Y:fLi = 1- 

1^.. We start with the following 

Lemma 4. Let a G (1, 2), i' > and tp he a real-valued random variable 
such that 



a = 1, 
a = 2. 



(49) 



E{exp{|^/;/z/|°}} < exp{l}. 



Then 
(50) 

t > ^ ln(E{exp{tV'}}) < t^{ip} + S{tvf + 2°"* a~^\tu\'^% a. 



a 



a — 1 



Proof. 1) Let t > be fixed. W.l.o.g. we can assume that v = 1. By 
Young inequality, we have 

a a* 

since a"^ (1/2)° < 1 and z> = 1, we have E{exp{a-i|V'/2|°}} < exp{a-^(l/2)"}, 
whence 

E{exp{t^}} < E{exp{a-V/2r + a;'(2*)"*}} < exp{a-i(l/2)" + a;i(2t)°*}. 

2) Let f{t) = E{exp{t'(/'}}. Since a > 1, / is a C°° function on the axis such 
that /(O) = 1, /'(O) = E{^} and 



/"(t)=E{exp{#}V^2| 
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It is easily seen that 

< i < 1/4 ^ exp{t\s\}s^ < exp{|s|°} Vs, 
whence under the premise of Lemma H] one has 

< t < 1/4 ^ f"{t) < exp{l} 
(recall that v = 1). It follows that 

< t < 1/4 ^ f{t) < 1 + tE{^p} + ?^^t^ < ew{m{i;} + ^^^^t^}. 
Thus, one has 

(a) 0<t<l/4^1n/(t) < tEW + ^t^^ 

Since St"^ > ^^^f and St"^ > a~^(l/2)" when t > 1/4, §B) implies pH) . 
SP.. Since a > 1, we have for all t > 

E {exp{t ELi i^j}} = E {exp{t E7=l Vj}E„-i{exp{tVn}}} 

< E{exp{exp{iEr=/V'i}}exp{//„t + 8(ti/„)2 + a-i2"*(ti/„)"*}, 

whence 

In (E{tEiIi V'.}} < + Sjvt' + Cjvt^, 

An = Ell = 8EiIi i^f, Cn = a-^2»' EIi ^t* ■ 

^ .. Recall that we are in the situation J2iLi — 1- We have for all t > 0: 

Prob{^Ar > Aiv + 71/} = Prob{exp{t^Ar} > exp{iyl7v + t7}} 

< E{exp{t^'7v}} exp{-MAr - ^7} < exp{57vt2 + Cat*^ 



whence 



Prob{^'Ar > ^AT +7} < inf exp{5Art^ + CATt— 1 - t-f}. 



whence also 

ln(Prob{^Ar > ^iv + 7}) < ln(2) + inf [ max[2SArt^ 2CArt^] -7*] = ln(2) - (?^*(7), 

t>0 V ^ y 

m 
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where is the Legendre transform of (p, Doixk^ = [0, cxd). Let = i*(a) be 
the unique positive root of the equation Bjyt'^ = CNt"* , that is, 



The function is strongly convex on [0, oo), equals 2B]\ft^ to the left of 
t* and equals 2C]yt°'* to the right of i*. Let 7_ = ^-{a) be the left, and 
7+ = 7+ (a) be the right derivative of at i*, so that 

1 

ABnU = 7- < 7+ = 2CNa^tr' ■ 

The function (/)*(7) is as follows: since (j) is strongly convex on [0, oo), 0'(O) = 
and (j){t)/t ^ cxD as t — > cxo, 0* is continuously differentiable and convex 
on [0,oo); when < 7 < 7_, 0* coincides with the Legendre transform 

~ shf^"^ °^ function IB^t"^ on the axis; when 7 > 7+, 

coincides with the Legendre transform (/>*^4.(7) = -^^ — ^ 7°^ of the function 

2Ciy\t\°'* on the axis. In the segment [7-, 7+] (/>* is linear with the slope 
(/)'^_„(7_) = (/)'^^+(7+) = t*. Now let 9 = _(7_ (7„), and let ^(7) = 
00*^+ (7). Observe that uj{'~f) < 4>^{'^) when 7 > 7_. 

Indeed, at the point 7-1- the functions 0,.+ and 4>* have equal values and equal 
derivatives, and since 0* is linear in A = [7-, 7+], we conclude from convexity 
of (?!)*,+(■) that 9!>.,+ (7) > 0.(7) on A, while < (t>'*,+{l) < 4>'*{l) = 0'*,+(7+) 
on A. Therefore 9 < 1, and since 0', is nondecreasing, we have uj'{'y) < 4''*(.'y) 
on A. Since (^{y-) ~ 4>*{l-), we conclude that uj < (j)^ everywhere on A. Since 
6 <1 and (f>*,+ is positive, when 7 > 7+ we have 0^(7) < 0t,+ (7) = 4'*{'y). 

The bottom line is that 




7" 



Recalling the definition of Ajy, Bjy. Cn, we arrive at (H7|) - (H8]l . 

4^.. We have proved the assertion of Proposition in the case of 1 < a < 
2. This combines with the standard approximation arguments to yield the 
assertion in the cases of a = 1 and a = 2. 



4.2.3. Completing the proof of Theorem \4.1\ 
f) . Preparations.. Given K-smooth space {E, || • ||), let us set 

^(^) = { fill L' 1 ;||e||>l ' Mi) = pyiUP) [/3>o], v{x) = \\ 

Observe that 



|2 
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1. Vg(-) is the Legendre transform of the restriction of Pv{-) on the || • ||*- 
unit ball, whence < 1 for all /? > and all ^, and 

(52) iix||, < 1 ^ (x,e) < Mx) + Vpio + Vf^ioy^ 



2. V{-) is continuously differentiable with — V'{r]) 

and is Lipschitz continuous, with constant 1, w.r.t. || • | 



* < - r]\ 



The second claim is evident. To prove the first, note that the 
function v{-) on the entire R" is strongly convex w.r.t. || • ||* with 
parameter k~^, whence, of course, so is the function v which is 
equal to v in the unit ball and is +oo outside of this ball. Given ^, rj 
and setting x = V'{£_), y = V'{ri), we have ^ e dv{£,), rj G dv{y), 
whence 

lie- y||* > {x~y,^~'n) > K-^\\x-y\\l, 

so that 

\\V'{0-V\y)\U^\\x-y\U<KU-y\\. 

3. One has 



(53) 



(a) \Vi3{^ + v)-Vf,m<\\v\\ 

{b) yp{c + v)-vpi()<{v;,{(),v) + §\\v 



|2 



It clearly suffices to consider the case of /3 = 1, that is, = V. 
By the second claim in item 2, V is Lipschitz continuous with 
constant 1 w.r.t. the norm || • ||, which implies (|53l a). Relation 
([551 b) is readily given by the Lipschitz continuity of V, see the 
first claim in item 2. 



; Proof of Theorem \4. 1\ (i) . . Let us fix /3 > and set 

n 

Sn = ^Ci, «n. = V^{Sn~l), i^n = Vgl'S'n) - Va(5„_i), 



i=l 



SO that a„ is jr„„i-measurable, and ^pn is ^„-measurable. By ([53l a) we have 
iV'nl < llCnII, whence 

(54) E„_i {eMl^nr/cT'^}} < exp{l}, 

while by (j53[ b) we have 

En_l {V'n} < E„_i {(a„,e„) + ^llCnf } = E„_i {(a„,e„) + ^llCnf } 

= E„_i l^ll'^np} [since an is J^„_i-measurable and E„_i = 0] 
< ^c72exp{l}. 
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The concluding inequality above can be justified as follows: setting = 
llCnII/fn, we have E„_i {exp{C"}} < exp{l}. At the same time, it is imme- 
diately seen that 

< (aexp{l}/2)-2/"exp{|s|"} 

for all s, and since (aexp{l}/2)-2/" < Iwhenl < q < 2, we get E„_i{C^} < 
E„„i {exp{|<^„|"}}. Thus, we arrive at 

(55) E„_i {ipn} < /in := exp{l}fT^. 

Invoking ([52]) . we get 

\\SN\\<^ + V^iSM) = '-^ + Y.^i. 

i=l 

Taking into account (j54p . ()55p and applying Proposition 14. H we arrive at 



V7 > : Prob<^ \\Sn\\ > 



§_ _|_ Kexp{l}^.^^g-: 
2 2/3 



^2 



+ 7vSir^} <2exp{-imin[72,7r"7"]}, 



with 7* = 7^.(q, cr^) given by ([15|) . Optimizing this bound in /? > 0, we 
arrive at (j42p . Theorem 14. l[ (i) is proved. 

5*^.- Proof of Theorem These results are given by exactly the 

same reasoning as above, with the role of Proposition 14.11 played by the 
following statement: 

Proposition 4.2. Let tpi, ^ = 1, , he Borel functions on Vt such that 
ipi is Ti-measurahle, and let /Xj > 0, > he deterministic reals. Assume 
that almost surely one has 

: < fii, 

and either 

(56) Vi : Ei„i {exp{^f/uf}} < exp{l}, 
or 

(57) Vi : l-^il < Vi. 

Then for every 7 > one has 
(58) 



N N 



Prob V > V u- + -Y V 1/2 I < / «^^P{-7V3}, case of ([56 
\t{ t^,^ \h \ \ exp{-7V2}, case of §. 
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Proof. Let (I56p be the case. It is immediately seen that exp{s} < s + 

4 

3ui ■ 



exp{9s^/16} for all s. We conclude that if < t < then 



Ei_i {exp{t^,}} < t/^i + E,_i {exp{9tV»Vl6}} 

(59) < tfii + exp{9tV,Vl6} < exp{t^i + 9fiyf/l6}. 

Besides this, we have tx < ^^^-^ + 1^, so that 

Ei_i {exp{t^i}} < exp I + - L 

and the latter quantity is < exp( — ^) when t > 3^. Invoking ([59]) . we 
arrive at 

(60) t > ^ E„_i {exp{t0„}} < exp{t^f, + St'^u^/A}. 
It follows that 

Eexp{t ELi ^i} = E{E„_i {exp{t E^=i < E {exp {tE7=l V'i}} exp(t/x„ + 3t^i^l/4), 

whence 

t > ^ E {exp{t Eili V'^}} < exp {tj:f=i + '4 EIi ^t] ■ 
Therefore for 7 > we get 



Prob Ef=i V'i > Ef=i + 7Vi:f=i 



< 



mmt>o 



E {exp{t Eili ^*}} exp{-t Eili " } 



< mini>o exp{t Ef=i /^i + ^ Ef=i " * Ef=i " ^ VEf=i ^^f} = exp{-7V3} 

as required in the first bound in ()58p . In the case of ()57p . by Azuma- 
Hoeffding's inequality [1], we have 

Vt > : E„_i {exp{i0j} < exp{i/ii + ^2/2}; 

with this relation in the role of (j60p . the above reasoning results in the 
second bound in (l58l). ■ 
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